Skip to content

Phase 2, theme 3: async ChoicesFrom lookups and worker ownership (ASY-01..08), Tools compatibility at load, conformance tests - #32

Merged
guijacquemet merged 5 commits into
mainfrom
claude/phase2-qupath-async-compat
Oct 13, 2026
Merged

Phase 2, theme 3: async ChoicesFrom lookups and worker ownership (ASY-01..08), Tools compatibility at load, conformance tests#32
guijacquemet merged 5 commits into
mainfrom
claude/phase2-qupath-async-compat

Conversation

@guijacquemet

@guijacquemet guijacquemet commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Theme 3 of the wrong-science wave for QuPath: asynchronous ChoicesFrom lookups and who owns the worker (SPEC_WRONG_SCIENCE section 5, ASY-01..08), a Tools compatibility check when the registry is read, and the conformance tests that read conformance.json (docs/CONFORMANCE.md says QuPath has tests/gui_test_conformance_body.groovy; it did not exist). Reference: Napari #54.

Fixes #19.

This PR is based on PR #31 (claude/phase2-qupath-number-limits), which is based on PR #30 (claude/phase2-qupath-retention): it needs the parts.txt split, the runs test app and the new run_gui_ci.sh entries of those two. Merge #30 as a merge commit, retarget #31 to main and merge it the same way, then retarget this one.

What changes

Async lookups and worker ownership (issue #19)

Before: resolveChoices started one thread per ChoicesFrom parameter; each ran ensureWorker (read and overwrite the shared worker / workerApp fields, no lock) and worker.run; a run did the same from its own thread; choiceSeq was reset per form, so a late answer of tool A with number 1 could fill the dropdown of tool B (also number 1); and a lookup could be running on the worker a run was starting on.

  • LcLookups (LcLookups.groovy): every question is (form generation, app, tool, parameter, the values it depends on, request id); the generation is process-wide and never reset (it grows when a form is built or destroyed), the request id is per window and never reset. An answer is applied only if all of it still holds when it arrives (the dependencies are recomputed then); otherwise it is dropped whole, with its error and tooltip. A dependency change invalidates at once (the dropdown goes back to the text field with the tooltip "waiting for the choices...") and only the new question waits for the 400 ms pause. One lookup in flight per window on one thread; waiting questions are coalesced (one slot per parameter); at most 8 wait (the oldest is dropped and logged); a lookup past 60 s is ended with its worker (-Dlc.qupath.lookup_timeout_s shortens it for tests, said in the status as did not answer within N s).
  • LcWorkers (LcWorkers.groovy): the one owner of the workers of a window, under one lock. A run or a lookup leases a worker when it is submitted and uses only that one. A lookup never uses the worker a run holds and never starts while a run is active (it asks again after 1.5 s); a run waits for a lookup that holds its worker. An app switch during a run does not replace the run's worker (the new app's worker is made for the next job; the old one is closed when its last job ends). A crashed or timed-out worker is discarded. Closing the window drops the waiting lookups, ends the one in flight, stops the thread and closes every worker (also one that was being made at that moment). LcWorker implements LcWorkerLike, counts the processes it starts and takes a time limit; dlg.worker / dlg.workerApp are now read-only views of the controller; Cancel goes to the worker of the run (the run's lease).

Compatibility when the registry is read

LcCompat (introduced in PR #30 for the run-time refusal) now also runs in LcRegistry.load(): an app whose Tools is older than 0.3.0a1, or lacks reserved_inputs_v1 / retention_prune, or has no version, is not offered and is listed under "skipped" with one sentence (... is older than the 0.3.0a1 this host needs: update it); an unparsable version is offered with a warning in the status line. README has a "Tools compatibility" section.

Conformance tests

gui_test_conformance_body.groovy (115 checks): validates the file with Tools' --validate; compares every limit, message template and host-internal code of the file with this script's constants (an unknown key fails: a new rule is a failing test until the host follows it); runs all 7 compat vectors and the version order through LcCompat; runs the done-status rule through the real Tools helper for each way a run can end; and keeps a ledger of every fixture id of the file (83) mapped to the body that runs it or to the reason it does not apply (printed as UNSUPPORTED, never passed): CF-02..06, CF-13 (no transform), RET-01..09, 12, 13 (Tools' own), RET-14, RUN-05.
gui_test_async_body.groovy (64 checks): ASY-01..08 read from conformance.json against gated source tools of the runs test app (the dependency of the dropdown is the gate path, so the test controls both the delay and the dependency): see README. ASY-08 also runs three threads over fake workers at a fixed interleaving (a barrier hook) and at random.

Existing tests changed (spec supersedes them)

  • gui_test_area_body.groovy: fx { dlg.worker = null } (it assigned the field) is dlg.stopWorker(true).
  • gui_test_parity_trust_body.groovy: its good schema now says tools_version and features (an app with an older Tools is not offered); new cases for the compatibility rules.
  • gui_test_retention_body.groovy (from PR Phase 2, theme 1: retention markers + prune subprocess + Tools run record, and run identity (RUN-01..07) #30): its Tools-too-old block registered three apps with an old Tools and ran them; they are now not offered when the window rescans (the body checks the status text and that they are not in dlg.apps), and the run-time refusal is checked by putting such an app into the window's list.
  • tests/runs_app gained the gated source tools; tests/run_gui_ci.sh lists the two new bodies.
    Nothing was skipped or loosened.

Sabotage checks

(see the comment on this PR)

Spec gaps and conservative choices

  • Section 5 says a lookup that exceeds 60 s is "cancelled and its worker discarded"; here it is ended together with its worker (a cancel request would need a grace period that this thread cannot wait for), and the status says so. The 1 / 8 / 60 s limits are status: default in conformance.json.
  • ASY-04's "at most 1 pending per parameter" holds by construction (one slot per parameter); the test checks the total and the dropped count with 12 parameters at once.
  • A run waits for a lookup that holds the worker it needs (a lookup is short and limited to 60 s); the spec does not say which of the two waits.
  • Closing the window blocks the FX thread only while a worker is being made (a few tens of milliseconds); a worker made while the window closes is closed.
  • The compat result "warn" only appears for a version text that is not understood (a missing version is a refusal here because this host sets a minimum).
  • RET-01..09, 12, 13 are Tools' own fixtures; RET-14 and RUN-05 do not apply; CF-02..06, 13 do not apply (no transforms).

The other open issues

Not verified

  • Windows and macOS (process handling, ProcessHandle, the file locks of Tools' prune); no human at a display (private Xvfb, real QuPath 0.7.0 / Java 25). Timing: the retry after a run is 1.5 s and the tests wait for it; on a very slow machine a test could time out, not fail wrongly.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN

Generated by Claude Code

claude added 4 commits October 13, 2026 17:58
@claude
Async ChoicesFrom lookups and worker ownership, Tools compatibility a…
91c7a2e 
…t load, conformance tests

LcLookups: every lookup is identified by (form generation, app, tool, parameter, dependencies, request id) and
its answer is applied only if all of it still holds; a dependency change invalidates at once; one lookup in flight,
coalesced questions, at most 8 waiting, a time limit. LcWorkers: one owner for the workers of a window; a job
leases a worker and uses only that one; a lookup never uses or replaces a run's worker; closing leaves no worker.
LcCompat runs when the registry is read. New bodies read conformance.json (limits, messages, codes, compat vectors,
done status, a ledger of every fixture id) and run ASY-01..08.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
@claude
Async test: each of the five parts of a lookup's identity is checked …
6a9e2e8 
…on its own

A reply that differs from the current question in only the form generation, the app, the tool, the
request id, the dependencies or the parameter is dropped; the same reply with all of them current is
applied. (Found by sabotage: dropping the generation check alone was not caught, the other checks
covered it in every fixture.)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
@claude
Merge branch 'claude/phase2-qupath-number-limits' into claude/phase2-…
345800e 
…qupath-async-compat

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
@claude
Async test: a filled dropdown goes back to pending at once when its d…
0336ab7 
…ependency changes

Found by sabotage: removing the visible part of the pending state (the dropdown cleared, the text field with its
tooltip shown) was not caught, because in the delayed-source fixtures the dropdown had never been filled.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
@guijacquemet Claude

guijacquemet commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Sabotage checks

Each row: production code broken in a copy of the final tree of this PR (a regular expression replaced one place), the PR's own body run on it (gui_test_async_body; gui_test_conformance_body for the limits, messages, compat, done-status and ledger rows; gui_test_parity_trust_body for the registry row). Every row fails the body except the one marked NOT DETECTED, which is an equivalent mutant: a leased worker is always either the kept one or a retired one, and closeAll closes both, so removing the third source of workers changes nothing observable. Three rows exposed a hole in the first version of the tests and the tests were extended (two further commits on this branch): removing only the form-generation check was not caught (the id, tool and dependency checks covered it in every fixture; now each part of the identity is tested alone), and removing the visible half of the pending state was not caught (the dropdown had never been filled in those fixtures; now a filled dropdown is tested).

sabotage (production code broken in a copy) what the test run said
a reply is applied whatever its identity (no check at all) 4 failed check(s): ASY-01_A_s_reply_was_dropped, ASY-02_the_old_reply_is_dropped, ASY-02_it_is_still_pending_and_shows_no_choices_after_the_old_reply ...
the form-generation check alone is removed (found as NOT detected by the first version of the tests; the identity unit checks were added) 6 failed check(s): ASY-02_a_reply_of_another_form_generation_is_dropped, ASY-02_a_reply_of_another_app_is_dropped, ASY-02_a_reply_of_another_tool_is_dropped ...
a dependency change does not mark the parameter pending 2 failed check(s): ASY-02_the_parameter_is_pending_at_once, ASY-02_it_is_still_pending_and_shows_no_choices_after_the_old_reply
the pending state keeps the old dropdown on screen (dropdown not cleared, text field not shown) 1 failed check(s): ASY-02_a_filled_dropdown_is_pending_and_empty_at_once_when_its_dependency_changes
a dependency change waits for the debounce before invalidating (no immediate invalidation) 2 failed check(s): ASY-02_the_parameter_is_pending_at_once, ASY-02_it_is_still_pending_and_shows_no_choices_after_the_old_reply
the limit of 8 waiting lookups is raised to 100 2 failed check(s): ASY-04_at_most_8_lookups_wait_and_the_oldest_are_dropped, ASY-04_the_drops_are_logged_oldest_first
lookups run in parallel (a thread per lookup) 2 failed check(s): ASY-04_at_most_one_lookup_in_flight, ASY-05_never_more_than_one_lookup_at_a_time
the worker controller gives a lookup lease while a run is active 2 failed check(s): ASY-08_no_lookup_lease_while_a_run_is_active, ASY-08_the_retired_worker_is_closed_when_the_run_ends
an app switch closes the worker a run still holds 2 failed check(s): ASY-08_an_app_switch_does_not_close_the_worker_a_run_holds, ASY-08_stress_nothing_failed_and_every_worker_was_closed_exactly_once
a run does not wait for the lookup that holds its worker 1 failed check(s): ASY-08_a_run_waits_for_the_lookup_that_holds_its_worker
a worker can be closed twice test aborted with an exception (rc=1 after 35 checks)
closeAll does not close the leased workers (the ones neither kept nor retired) NOT DETECTED
closing the window does not stop the lookup thread 2 failed check(s): ASY-07_the_lookup_thread_stops, ASY-07_the_lookup_thread_of_that_window_stops_too
leases are still given after the window is closed 1 failed check(s): ASY-08_no_lease_is_given_after_the_close
the time limit of a lookup is ignored 3 failed check(s): lookup_over_the_limit_is_ended_and_said, lookup_over_the_limit_discards_its_worker, lookup_after_the_timeout_works_on_a_new_worker
the pending limit is 9 instead of 8 (the conformance value) 1 failed check(s): limit_lookup_pending_max_is_the_conformance_value
the target_unavailable sentence is reworded 1 failed check(s): message_target_unavailable_is_the_conformance_template
the below-minimum rule of the compatibility check is removed 1 failed check(s): compat_below_the_minimum_is_refuse
the record helper is never told that Cancel was pressed 1 failed check(s): done_status_crashed_after_cancel_is_the_worker_killed_after_the_grace_period_is_cancelled
a fixture id of conformance.json (ASY-08) is missing from the ledger 2 failed check(s): ledger_ASY-08_is_classified, ledger_the_file_has_the_fixture_counts_this_host_expects
the registry accepts apps with an old Tools 4 failed check(s): trust_tools_old_refused, trust_tools_no_feature_refused, trust_tools_no_fields_refused ...

"aborted with an exception" means the body died on a missing marker or on the stress threads (the exception fails the body).

Generated by Claude Code

@claude
Retention body: an app with an old Tools is not offered any more; the…
3cbc480 
… run-time refusal is checked on an app that is in the list anyway

With the compatibility check in the registry load, the three old-Tools apps of the body are skipped when the
window rescans (the status says why); the run-time check stays as the last line and is exercised by putting
such an app into the window's list.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
@guijacquemet
guijacquemet changed the base branch from claude/phase2-qupath-number-limits to main October 13, 2026 19:02
@guijacquemet
guijacquemet merged commit 646cdc4 into main Oct 13, 2026
2 checks passed
@guijacquemet
guijacquemet deleted the claude/phase2-qupath-async-compat branch October 14, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Serialize ChoicesFrom lookups and protect QuPath worker ownership across app/tool changes

2 participants

@guijacquemet @claude